Useful Links - Performance

< >

RidgeRun Video Stabilization Library

Table of Contents [Sticky]

• Basics and Foundation
  • What is Video Stabilization
  • Video Stabilization Process using IMU
  • Stabilization Algorithms with IMU
• Getting Started
  • Supported Platforms, Sensors and Backends
  • Evaluating the Library
  • Getting the Code
  • Building the Library
• API Reference
  • Library Architecture
  • Adding New Sensors
  • Adding Algorithms
  • Adding Stabilization Algorithm
  • Adding Backends
  • API Documentation
  • Examples Guidebook
• Library Integration for IMU
  • Preparing the Video
  • Preparing the IMU Measurements
  • Measurement Integration
  • Interpolation
  • Computing the Stabilization
  • Video Undistortion
  • Example Application
• GStreamer
  • Add Support for Video Timestamps
  • Add Support for IMU Sensors
  • Video Stabilizer Element
  • Example Pipelines
• Useful Links
  • BMI160 Setup
  • Performance
• Contact Us

Performance

The heaviest stage of the algorithm is the Video Undistortion. The following metrics come as a result of measuring this stage.

Overall Latency

The RidgeRun Real-Time Video Stabilizer has a latency of the processing time of one single frame, it is important to consider that this is the stabilizer process latency, you need to consider possible additional latency added by: video dequeue, pipelines queues, encoding/decoding process, streaming or another process required by your project.

Qualcomm RB5

This is using a synthetic image running at average conditions on Ubuntu 20.04 during 100 iterations.

Backend	Dimensions	Processing Time (ms)	Average Framerate (fps)
OpenCV	720p	22.28	44.88
	1080p	171.6	5.83
	4K	781.7	1.28
OpenCL	720p	11.00	91.00
	1080p	18.69	53.5
	4K	41.65	24.01

It used the concept example for performance measurement. The CPU load should be similar to the Xavier NX, consuming up to 2-3 cores.

NVIDIA Jetson

Xavier NX

This is using a synthetic image running at average conditions on Jetpack 5.1.2 during 100 iterations. It uses the maximum performance power profile and jetson clocks. The capture element is the NvArgusCameraSrc at 30 fps (the estimated fps are derived from the processing time from GstShark). The CPU load is the percentage of a single core usage using perf and the GPU load using tegra stats.

Backend	Dimensions	Processing Time (ms)	Average Framerate (fps)	CPU Load (%)	GPU Load (%)
OpenCV	720p	61.55	16.26	50	0
	1080p	195.59	5.11	50	0
	4K	979.83	1.02	50	0
CUDA	720p	10.02	99.84	31.13	4.68
	1080p	12.35	31.95	32	22.22
	4K	24.03	41.62	40.38	46.43

Orin AGX

This is using a synthetic image running at average conditions on Jetpack 6.0 during 100 iterations. It uses the maximum performance power profile (MAXN) and jetson clocks. The example concept is used to compute the maximum framerate achieved (without considering memory transfers since CUDA Memory Managed is assumed). The results for CPU and GPU will come soon.

Backend	Dimensions	Processing Time (ms)	Average Framerate (fps)	CPU Load (%)	GPU Load (%)
OpenCV	720p	20.59	48.57	-	-
	1080p	50.03	19.99	-	-
	4K	229	4.37	-	-
CUDA	720p	0.84	1190.5	-	-
	1080p	1.371	729.30	-	-
	4K	5.65	176.89	-	-

NXP i.MX8 MP

This uses a synthetic image running at average conditions on Yocto Hardnott during 100 iterations. Please, note that the imx8mp is one of the most basic boards.

Backend	Dimensions	Processing Time (ms)	Average Framerate (fps)
OpenCV	720p	84.42	11.85
	1080p	192.77	5.19
OpenCL	720p	85.36	11.72
	1080p	169.32	5.91

X86 PC

This uses a synthetic image running at average conditions on Ubuntu 18.04 during 100 iterations.

• CPU: Intel i7-1065G7 (8) @ 3.900GHz
• GPU1: NVIDIA GeForce MX230
• GPU2: Intel Iris Plus Graphics [0x8a52]

Backend	Dimensions	Processing Time (ms)	Average Framerate (fps)
OpenCV	720p	26.65	37.52
	1080p	38.85	25.74
	4K	922.1	1.08
OpenCL (NVIDIA MX230)	720p	18.22	54.88
	1080p	26.84	37.26
	4K	50.1	19.96
OpenCL (Intel Iris)	720p	6.53	153.14
	1080p	8.59	116.4
	4K	17.54	57.01
CUDA (NVIDIA MX230)	720p	17.58	56.25
	1080p	25.7	38.91
	4K	78.9	12.67

< >